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Abstract 

A common method for texture representation is to 
use the marginal probability densities over the out- 
puts of a set of multi-orientation , multi-scale filters 
as a description of the texture. We propose a tech- 
nique, based on Independent Components Analysts , for 
choosing the set of filters that yield the most infor- 
mative marginals , meaning that the product over the 
marginals most closely approximates the joint proba- 
bility density function of the filter outputs. The al- 
gorithm is implemented using a steerable filter space. 
Experiments involving both texture classification and 
synthesis show that compared to Principal Components 
Analysis , ICA provides superior performance for mod- 
eling of natural and synthetic textures. 

1 Introduction 

One of the main goals of computer vision is the com- 
pact representation of visual entities. Textures, the 
visual objects considered in this paper, are best repre- 
sented by their statistical properties. We introduce an 
information-theoretic framework that provides a bet- 
ter understanding of filter-based approaches to tex- 
ture analysis and leads to a technique for improving 
the quality of those texture representations that are 
based on marginal statistics over filter outputs. 

Loosely speaking, textures are characterized by two 
basic properties: homogeneity and locality of repre- 

sentation. In other words, the “visual flavour 1 ’ of 
a texture can be captured by its statistical behav- 
ior within a limited size window. Suitable statistical 
models for textures are stationary Markov Random 
Fields (MRF), which are completely characterized by 
their joint probability density function (pdf) within 
a suitable neighborhood. Estimating and represent- 
ing even low-dimensional joint pdf s. however, can be 
an overwhelming task. The size of the representation 
grows exponentially with the model dimension, as does 
the minimum number of samples required to achieve 
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a given estimation accuracy [8]. In fact, techniques 
that attempt to completely characterize joint distri- 
butions either consider very small neighborhoods [25], 
measure only pairwise dependency [15], or use specific 
MRF models [10], [6]. 

A different approach is based on the analysis of 
feature vectors formed by the output of a filter 
bank [18], [7], [19], [21]. With a suitable choice of the 
analysis filters, we can assume that the feature vectors 
capture the local visually significant texture charac- 
ter. In other words, the filters map the image values in 
each neighborhood onto a “perceptually relevant ’ sub- 
space, thereby reducing the representation size while 
preserving structural information. 

Barring a few exceptions [25], [11], [28], typical filter- 
based algorithms do not estimate the complete statis- 
tical description of the texture features. Instead, they 
build models based on the marginal statistics of the 
feature components, usually represented by the chan- 
nel variances or their histograms. For example, by 
analyzing the directional characteristics of a texture 
along a dense (ideally continuous) set of orientations 
and scales, one obtains its “scale/orientation signa- 
ture 11 , which can be used for classification [21]. [27]. 

Texture representation by marginal statistics is a 
simple and attractive approach. To make efficient use 
of it, though, we should understand how well a given 
set of marginals represents the joint statistical descrip- 
tion of a texture feature. Zhu et al. [31] pointed out 
that, in general, one needs the marginal pdf’s along ev- 
ery possible projection direction to completely charac- 
terize the joint feature vector pdf. Thus, one approach 
to improving the quality of representation is to use a 
large number of filters, perhaps exploiting steerable 
schemes [13], [24] for computational efficiency. 

In this work we follow a different route, and de- 
vise a technique for finding the basis of a given filter 
space which generates the most informative marginals 
for a given texture, meaning that the product over 
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the marginal densities most closely approximates the 
joint pdf. We show by experimental tests of classifi- 
cation and synthesis that by selecting such “optimal” 
filter basis the quality of the representation increases 
significantly. 

Our basis selection algorithm is based on Indepen- 
dent Component Analysis (ICA) theory. ICA has been 
an area of intense research during the last decade, 
stimulated by problems of blind source separation and 
deconvolution. ICA can be regarded as a technique for 
statistical modeling that proves superior to Principal 
Component Analysis in the case of non-gaussian ran- 
dom vectors. It is an established notion that natural 
images do not behave like gaussian processes [20] [3]; 
our experiments show that ICA provides better mod- 
els for natural and synthetic textures. 

Other filter selection techniques have been proposed 
for texture analysis, usually based on energy [5] or 
class separability [12]. Perhaps closest to our approach 
is the entropy-based algorithm of Zhu et al. [31], 
which uses a greedy strategy to sequentially introduce 
one filter at a time. 

This paper is organized as follows. Section 2 de- 
scribes the filter spaces that we use for texture anal- 
ysis, and introduces the filter basis selection problem. 
Section 3 provides some necessary background of ICA, 
and shows its application to texture modeling. Sec- 
tion 4 describes texture classification and synthesis 
experiments from which we were able to assess the 
performance of ICA modeling. Section 5 has the con- 
clusions. 

2 Steerable spaces for texture analysis 

Given an image l(x) and a set of N filters 
{/i,(x)}, we will say that the vector f(x) — 
[fx(x)=l*hi{z) t ... t fN{x) = l*h N [z)] is the fea- 
ture representation of the image at point x. We will 
assume that a texture is completely represented by the 
joint pdf of its feature vector f(x) (which, by station- 
ariety, is independent of x.) By this we mean that 
two textures with the same joint distribution of their 
feature vectors are "‘perceptually” indistinguishable. 

W'hich filters produce meaningful feature represen- 
tations of textures? Several design criteria have been 
proposed. Basically, they all share the following char- 
acteristics: I) zero-DC response, to enforce invari- 

ance to slow-varying illumination bias; 2) good spa- 
tial/spectral concentration, to ensure energy separa- 
tion while preserving locality of description. Another 
useful property of analysis filter banks is steerability 
[13], [14] defined as follows. Let H be the linear space 
spanned by the N kernels {/i,(x)} t and let Q be a suit- 
able group of domain transformations (e.g., rotations 


or isotropic scaling.) We will say that H is steerable 
over Q if for every kernel h{x) in H , and for every 
transformation A() in Q , the ‘"transformed” kernel 
/^(x) = k(A(x)) belongs to H. For example, assume 
the filter space is steerable over the group of rotations. 
Let f(x) and / H (x) be the feature vector description 
of a texture and of the rotated version of the texture, 
respectively (/Z is the rotation matrix). Then, f(x) 
and f R (R~ 1 x) are related by a linear transformation 
(a matrix), which is a function of the rotation angle 
only. Steerability, thus, ensures uniformity of repre- 
sentation for transformed versions of the same tex- 
ture. Methods exist to design exact and approximate 
finite dimensional steerable filter spaces over rotations 
and isotropic scaling over a finite scale range of inter- 
est [24]. 

Joint spatial/spectral concentration is achieved, for 
example, using Gabor kernels or directional deriva- 
tives of gaussian kernels [19], [21]. Such filters effec- 
tively capture directional texture attributes. If a TV- 
dimensional steerable analysis space is used, the tex- 
ture signatures are completely characterized by the 
joint statistics of the output of N basis filters. Note 
that one can always find a set of basis filters which 
are scaled/rotated versions of a prototype kernel in 
the steerable space. 

In this paper we study reduced representations 
formed by the marginal statistics of the feature vec- 
tors. Let us recall that the marginal density of the 
i-th component of a random vector c, p<(x<) f is the 
projection of the joint pdf p(r) onto the i-th axis. 
The set of marginal statistics is represented by the 
outer product of the marginal pdf’s: p(z) = niP*(-*)> 
which is equal to p(~) if and only if the components 
of the feature vector are statistically independent. In 
particular, we are concerned with the selection of the 
“optimal” basis in a steerable filter space. Note that 
the choice of the basis is irrelevant if the joint pdf 
of the feature vector is considered. Indeed, if fa and 
fa are the feature vectors at point x, computed using 
two different filter bases, then fa = Af i, where A is a 
full-rank matrix. Thus, the joint pdf's of the feature 
vectors, pi(z) and po(::), are related to one another as 
po(z) = pi(.4“ l ::)/| det(.4)| [23]. This one-to-one re- 
lation, however, holds for joint pdf’s only. In general, 
the marginal pdf's of f\ cannot be computed from the 
marginal pdf's of fa. This somewhat paradoxical ob- 
servation stems from the fact that marginal pdf's are 
projections of a joint pdf along different directions. A 
finite set of marginals does not carry enough informa- 
tion in general to allow reconstructing all the other 
projections. 



Assume, however, that a particular choice of basis 
filters gives statistically independent feature compo- 
nents. Then, as noted above, the marginals completely 
characterize the joint pdf of the feature vector: from 
this set of marginals, all the other marginals can be 
reconstructed. The converse is not true: other filter 
bases will produce, in general, features with “less in- 
formative” marginal statistics. 

Unfortunately, only very few textures can be repre- 
sented by independent component features [3]. Still, 
it is almost always the case that some filter bases pro- 
duce “more informative” marginals than others. This 
intuitive notion can be made more rigorous using the 
theory of Independent Components Analysis (ICA); 
ICA also leads to an algorithm for selecting an "opti- 
mal” filter basis. 

3 Independent Component Analysis 

In this section we report some results of Indepen- 
dent Component Analysis theory that are instrumen- 
tal to the development of the texture classification and 
synthesis algorithms discussed in later sections. We 
refer the reader to the excellent tutorial of Cardoso [4] 
for a general overview of the theory. 

We first introduce the ICA problem as a statistical 
approximation tool, and then briefly outline Comon’s 
ICA algorithm, which we have adopted in our experi- 
ments. We also present a simple case study that shows 
the effectiveness of ICA modeling for texture analysis 
in a steerable space. 

3.1 Problem formulation 

The theory of Independent Component Analysis is 
traditionally associated with the Blind Source Sepa- 
ration (BSS) problem. In its simplest formulation, 
the BSS problem assumes that N independent causes 
(random variables) have been linearly combined by a 
full rank matrix A to produce N observed variables. 
The goal is to identify the mixing matrix from the ob- 
servations, possibly using prior information about the 
statistics of the causes if available. 

ICA, however, may also be regarded as a general 
estimation method [9], without explicit reference to 
the BSS model. If z is a random vector with joint 
pdf p(z) and full-rank covariance matrix 1 , ICA seeks 
a full rank matrix *4 such that the pdf of the vector 
y = Az is “best represented” by the outer product 
of its marginal pdfs. The quality of the separable 
approximation is measured by the contrast , w'hich is a 
deterministic function of a joint pdf. Let e(z) be the 
'"model error”, defined as the difference between the 

1 If the covariance matrix V is not full rank, we may consider 
the projection of z onto the range space of V [9]. 


logarithm of the density p(z) and the log-likelihood of 
the separable model: 

e(z) = logp(r) - log (1) 

I 

where Pi{zi) are the marginal pdf’s of the components 
of z. The contrast [9] of p(z) is defined as the expec- 
tation of the model error e(z), which is equal to the 
Kullback-Leibler (KL) divergence between p(z) and 
the separable model pdf: 

£[e(i)1= /.„ pWlog rrab dz (2) 

i 

where //(*) represents the differential entropy of its 
argument: H(z) = - p(z) logp(z) (terms //,(*,) 
are called marginal entropies of z.) This contrast takes 
also the name of mutual information of z, and being a 
KL divergence it is always positive (it vanishes if and 
only if z has independent components). Intuitively, 
the mutual information tells us how much we lose in 
terms of average information if we neglect to consider 
the statistical dependence among the components of 
z. 

Other contrast functions have been proposed for 
ICA. For example, if the distributions gi[zi) = 
fl' Pi{z)dz of the original independent components 
in a BSS model are known, we may define the “info- 
max” contrast [1] of the vector z as — H(g(z)), where 
g(z) = \gi(z {)< . . . ,5Nv(zjv)]- However, in the case of 
texture descriptors we don’t know the distributions 
gi{z{), and mutual information is a more appropriate 
criterion. Thus, we define the ICA problem as follows: 

ICA problem: find a full-rank matrix A 
such that y = Az has minimal mutual infor- 
mation. 

Note that the solution to the ICA problem is not 
unique: if A minimizes the mutual information of Az, 
so do all matrices obtained by permuting the rows of 
A or by pre-multiplying A by a diagonal matrix. 

If z is jointly gaussian, its mutual information is 
null if and only if its covariance matrix is diagonal. 
Principal Component Analysis (PC A, also known as 
Karhunen-Loeve transform) is a technique to diago- 
nalize a vector covariance by an orthonormal matrix, 
and therefore solves the ICA problem for gaussian vec- 
tors. In the general non-gaussian case, however, diag- 
onal covariance is not sufficient to ensure the mini- 
mal mutual information condition; hence, more work 
is needed. 



It can be shown [9] that, if z has been “pre- 
whitened” to unit covariance (by PCA followed by 
axis rescaling), the ICA problem is solved by an 6r- 
thonormal linear transformation that minimizes the 
sum of the marginal entropies of the transformed vec- 
tor. This fact has a nice counterpart in the context of 
BSS. Since the gaussian distribution has the highest 
entropy for a given variance, we may argue that the 
demixing matrix is the one that produces marginals 
as “far from gaussian” as possible. This observation 
agrees with intuition: indeed, as suggested by the cen- 
tral limit theorem, linear mixing of independent causes 
produces gaussian-like distributions. 

3<2 Comon’s ICA algorithm 

ICA algorithms try to minimize the contrast of 
a vector without explicitly computing its joint pdf. 
Comon [9] proposed an efficient technique based on 
higher-order statistics. In fact, Comon’s algorithm 
minimizes a different contrast than mutual informa- 
tion; the relation between the two contrast functions 
can be highlighted using Edgeworth functional ex- 
pansions [17]. An intuitive justification of higher- 
order methods is based on the fact that all the cross- 
cumulants of an independent component vector are 
null [17]. Thus, one may try to minimize mutual in- 
formation by actually minimizing the n-th order de- 
pendence among the vector components. For exam- 
ple, Comon’s technique finds an orthonormal matrix 
A that minimizes the sum of the squared fourth-order 
cross-cumulants of y = Az , provided that z has been 
pre-whitened. It can be shown that this is equivalent 
to maximizing the sum of the squared marginal kur- 
tosis (fourth-order cumulants) of y. 

It is a well known fact that gaussian distributions 
have zero kurtosis [17]: Comon’s algorithm does push 
the vector components away from gaussianity. Lep- 
tokurtic, heavy tailed exponential-like distributions 
are characteristic of “sparse codes” [22], and are of- 
ten used to model the output of wavelet filter banks 
for image coding [20]. Platykurtic distributions are 
also often observed in texture analysis. 

Comon s design algorithm is based on Jacobi it- 
erations. Its complexity is 0{N A ) (including pre- 
whitening,) and it always converges to a (possibly lo- 
cal) minimum of the contrast. Due to these desirable 
properties, we have selected Comon's technique for our 
experiments. 

3.3 A case study 

In this section we present a simple example that 
shows the effectiveness of ICA-based modeling for tex- 
ture analysis. The two textures of Figure l (a),(b) are 
analyzed in the steerable space spanned by the two 


(0° - 90°) filters of Figure 1 (c),(d) (upper row). Tex- 
ture A is a mosaic of patches filled at random with 
either a constant signal, horizontal or vertical sinu- 
soidal gratings, or the sum thereof. Texture B is a 
mosaic of patches filled with either horizontal or verti- 
cal sinusoidal gratings. The 0° (90°) filter removes the 
horizontal (vertical) grating component completely. 

Our goal is to find two filters in the steerable 
space that best describe each texture by means of 
the marginal statistics of their outputs. Figure 1 
(c),(d) (lower row) show the filtered versions of the 
two textures using the 0° —90° filters. Figure 1 (e),(f) 
show the joint pdf’s of the filter outputs, measured 
by a Parzen window estimator. In both cases the 
two channels are uncorrelated with identical variance 
(as proved by the axial symmetries of their pdf’s.) 
However, only the channels of Texture A are statisti- 
cally independent. Thus, the outer product of their 
marginal densities reproduces the joint pdf of Tex- 
ture A exactly, as shown in Figure 1 (g). In the case of 
Texture B, though, the separable model yields a very 
poor approximation of the joint pdf (compare Figure 1 
(h) and (f)). As a matter of fact, the marginal pdf’s 
of the 0° - 90° channels of Texture B and Texture A 
are identical: based on such marginal descriptions, the 
two textures look the same! Note also that PCA can- 
not help us finding better bases, since the channels of 
both textures are already uncorrelated. 

If we run the ICA algorithm separately on each tex- 
ture (by analyzing the output of the two filters) we 
find that the 0° - 90° kernels are indeed the correct 
choice for Texture A, while the 45° - 135° kernels of 
Figure 1 (i) (upper row), obtained by multiplying the 
outputs of the 0°-90° filters by a 45° rotation matrix, 
are more appropriate for Texture B. Figure 1 (j) shows 
the outer product of the corresponding marginal pdf’s, 
after rotation back to the original 0° — 90° space. By 
comparing Figure 1 (f), (h) and (j) it is clear that ICA 
has dramatically increased the quality of the separable 
approximation. 

4 Experimental tests 

4.1 Supervised classification 

In this section we use ICA pdf modeling in a test 
of supervised texture discrimination. For the sake of 
simplicity, we perform training and classification on 
the same data set, formed by the I\ = 8 textures 
of Figure 2 (chosen from the MIT VisTex database). 
Each texture is modeled by the marginal pdf's over the 
selected channels in a fixed steerable space, together 
with the corresponding ICA matrix. The marginal 




Figure 1: A case study (see text.) (a),(b): Textures 
A and B. (c),(d): The 0° - 90° kernels (upper row), 
and the filtered versions of the two textures (lower 
row.) (e),(f): The joint channel pdf's of the two tex- 
tures. (g),(h): The separable approximations of the 
joint pdf's using the 0° —90° filters, (i): The 45° — 135° 
kernels (upper row’), and the filtered version of texture 
B (lower row.) (j) The separable approximation of the 
joint pdf of texture B using the 45° — Flo 0 filters. 


densities are estimated and represented by 50-bins his- 
tograms. 

An image formed by the mosaic of all textures is 
classified pixel- wise using a Bayesian technique [2]. A 
simple heuristic iterative procedure, inspired by the 
“Perceptually Organized Expectation Maximization” 
algorithm of Weiss and Adelson [30], estimates pos- 
terior class probabilities from the conditional likeli- 
hoods, enforcing local spatial coherence. The experi- 
ment proceeds as follows: 

• Training: for each texture model, 

1. filter the training image with a fixed filter 
bank; 

2. compute the ICA matrix from the output of 
the filters; 

3. multiply the output vectors by the ICA ma- 
trix and compute the channel histograms. 

• Classification: apply the fixed filter bank to the 
test image. For each texture model, 

1. multiply the filter output vectors by the 
model ICA matrix, and use the corre- 
sponding channel histograms to compute the 
marginal likelihoods; 

2. the likelihood of each pixel given the model 
is equal to the product of the marginal like- 
lihoods, times the absolute value of the de- 
terminant of the ICA matrix. 

Compute the posterior class probabilities and per- 
form pixel- wise Bayesian classification. 

For our experiment we have chosen a steerable pyra- 
mid filter bank [29] with just two orientations and 
three scales (N = 6 filters overall). For each train- 
ing texture (size 256 x 256 pixels), Comon's Matlab 
implementation of ICA required 27 seconds of compu- 
tation time on a Power Macintosh G3 266 MHz. The 
results, in terms of percentage of correctly classified 
pixels for each texture, are shown by the solid line 
plot of figure 2. If PC A is used instead of ICA, we 
obtain the correct classification rates shown by dot- 
dashed line; the classification performances of the sys- 
tem without transformation (i.e., using the original 
filters) are shown by the dotted line. ICA pdf model- 
ing consistently yields the highest correct classification 
rates. 

We have observed that the performance gain pro- 
vided by PCA and ICA modeling relative to the nomi- 
nal feature vector representation decreases if the num- 
ber of orientations in the steerable filter bank is in- 
creased. This should not come as a surprise: many 
marginal statistics can provide sufficient information 
even if they are not statistically independent; however, 
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Figure 2: Supervised classification test (see text). The 
system is trained and tested on the eight textures 
shown above. The plots represent the percentage of 
correct classification for each texture. Solid line: us- 
ing ICA transformation. Dot-dashed line: using PC A 
transformation. Dotted line: without transformation. 

the benefit comes at the price of increased representa- 
tion size. 

4.2 Texture synthesis 

Synthesis-by-analysis algorithms [16], [26], [31], [28] 
create new textures which “look like"’ a given proto- 
type. This is equivalent to sampling a random process 
whose statistical description has been estimated from 
a given sample. 

A simple and effective synthesis technique has been 
proposed by Heeger and Bergen [16]. First, the pro- 
totype texture is analyzed by a filter bank which in- 
cludes the “identity 15 filter, and the marginal channel 
histograms are recorded. Then, a random image is 
generated, and its channel histograms (computed by 
the same filter bank) are iteratively adjusted to match 
those of the prototype texture. The algorithm usually 
converges to a texture that has the same channel his- 
tograms as the prototype: the two textures are there- 
fore identical on the grounds of marginal statistical 
description. 

We have adopted such an algorithm as a testbed for 
our experiments because it represents the ideal “Tur- 
ing test' 1 to validate texture models based on marginal 
channel statistics. Our only addition to Heeger and 
Bergen's model is the multiplication of the channel 
vectors at the output of the filter bank by the ICA 
matrix computed for the prototype texture, as in the 
scheme of section 4.1. After the channel histograms 
have been adjusted to match those of the prototype 


texture, the channel vectors are multiplied by the in- 
verse of the ICA matrix. Note that the presence of 
the ICA matrix requires that all the channels be kept 
to the same sampling rate. 

Heeger and Bergen use steerable pyramid analysis 
filter banks [29], and synthesize samples that success- 
fully reproduce some unstructured characteristics of 
the prototype texture. Such simple marginal statistics 
modeling, though, fails to capture more complex spa- 
tial structures, such as elongated patterns. A simple 
way to boost the performance of Heeger and Bergen’s 
scheme is to enrich the filter space with the shifted 
versions of the filters. In other words, for each fil- 
ter h(x) we add the filters h n (x) = h[x + n) with 
n belonging to a suitable neighborhood of the origin. 
The feature representation thus “looks around” over 
a larger neighborhood of each point. This new infor- 
mation comes at no cost: the output of filter h n (x) 
at point Xo is equal to the output of filter h(n) at 
point xo 4- n. Shifted filters, though, are useless if the 
marginals statistical description is built directly from 
their output: by stationariety, the marginal pdf of a fil- 
ter output does not change if the filter is shifted. The 
contribution of the shifted filters becomes relevant if 
a different basis of the steerable space is chosen, that 
is, if the output of the filters is multiplied by the ICA 
matrix. 

We show in figures 3 and 4 examples of synthesis 
from the prototype textures “Crosses on clouds” and 
“Squares”. For the “Crosses on clouds” texture we 
have used a steerable pyramid filter bank which com- 
prised both odd-symmetric and even-symmetric ker- 
nels at two orientations and four scales (overall N = 16 
filters). For the “Squares” texture we have used odd- 
symmetric kernels at two orientations and three scales, 
shifted over five different positions (overall N = 30 
filters). These prototype textures are highly non- 
gaussian, which makes PCA modeling inadequate (see 
figure 3(c) and 4(c)). ICA modeling proves superior in 
both cases. In particular, for the “Crosses on clouds” 
texture, ICA does a good job at separating the low- 
pass, isotropic components (the “clouds”) from the 
line-shaped, oriented ones (the “crosses”.) In the case 
of the “Squares” texture, ICA is able to extract the lo- 
cal structure by correctly combining the shifted filters 
outputs. 

5 Conclusions 

We have presented an algorithm that chooses the 
basis filters in a steerable space in such a way as to 
yield the most informative marginals for texture repre- 
sentation. The method is based on the minimization of 
the mutual channel information via Independent Com- 





Figure 3: Examples of texture synthesis, (a) The pro- 
totype texture "Crosses on clouds'* is synthesized us- 
ing Heeger and Bergen’s technique (b) with no channel 
transformation , (c) with PCA transformation and (d) 
with ICA transformation. 



Figure -l: Examples of texture synthesis for the pro- 
totype text ure '‘Squares" (see rapt ion of figure (3)). 


ponent Analysis. The experimental results show the 
superiority of ICA modeling with respect to PCA for 
natural and synthetic textures and small dimensional 
filter spaces. 

An interesting open problem, which we are cur- 
rently investigating, is subspace selection based on 
information-theoretic criteria. We expect that by 
choosing a small number of highly informative chan- 
nels, one may obtain reduced size representations that 
maintain good discrimination properties. 
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